CPG-Actor: Reinforcement Learning for Central Pattern Generators

Central Pattern Generators (CPGs) have several properties desirable for locomotion: they generate smooth trajectories, are robust to perturbations and simple implement. However, notoriously difficult tune commonly operate in an open-loop manner. This paper proposes a new methodology that allows tuning CPG controllers through gradient-based optimisation Reinforcement Learning (RL) setting. In particular, we show how CPGs can directly be integrated as the Actor Actor-Critic formulation. Additionally, demonstrate this change permits us integrate highly non-linear feedback from sensory perception reshape oscillators’ dynamics. Our results on locomotion task using single-leg hopper explicitly rather than part of environment significant increase reward gained over time (20\(\times \) more) compared with previous approaches. Finally, our closed-loop progressively improves hopping behaviour longer training epochs relying only basic functions.